Clustering, multicollinearity, and singular vectors

Let A be a matrix with its Moore-Penrose pseudo-inverse † . It is proved that, after re-ordering the columns of , projector P = I − has block-diagonal form, that there permutation Π such T diag ( S 1 2 … k ) further each block i corresponds to cluster are linearly dependent other. clustering algorithm provided allows partition into clusters where in correlate only within same cluster. Some applications supervised and unsupervised learning, specially feature selection, clustering, sensitivity solutions least squares discussed.